1. Field of the Invention
The present invention relates generally to a trace processing program, method and apparatus for a distributed application system achieving application services via a network and, more particularly, to a trace processing program, method and apparatus for collecting traces and identifying a bottleneck point which has caused degradation of responses.
2. Description of the Related Arts
In a network-enabled distributed application system, a plurality of servers is configured on multilayer, and applications operated thereon are also intricately combined wherein one (1) service is achieved by operating each one coordinately. In such complicated distributed application system, in order to identify a bottleneck point which has caused degradation of responses, research and analysis are performed by, for example, putting in chronological order and linking trace information in which behavior of an application is recorded. Collection of trace information is recorded and accumulated at an entry point when the executed application (including a program or a module in the program) is invoked and an exit point when a process is terminated, with time information when each point is passed through.
FIG. 17 illustrates conventional collection of trace information, wherein a web server 200, an application server 202, a database server 204 and a database 206 are arranged in hierarchical configuration, and a request telegraphic message from a client is processed by executing an application of each server and responded by, for example, searching the database 206. The trace information is collected at point P1 to P12 which are entry points and exit points of the executed applications. If a transaction response t1 is degraded, a bottleneck point can be identified by comparing transaction responses w1, w2, a1, a2, d1 and d2 of the servers obtained from the trace information with those of normal time. In order to facilitate linking of trace information for checking what calling-relationship each intricately combined application has cooperated in, some systems are ingeniously designed about recorded information, wherein, for example, not only the time information of entry and exit points, but also information of a next-invoked application is recorded, as shown in FIGS. 1A and 1B. FIG. 1A shows an example of a transaction distribution, and correspondingly, as shown in FIG. 1B, calling-hierarchy information of each application is generated (see, e.g., Japanese Patent Application Laid-Open Publication Nos. 1989-277940, 1992-84234 and 2001-318809).
However, a trace processing method of a conventional distributed application system has following problems. First, the conventional distributed application system has too large overhead. In a distributed application system, information subject to tracing, such as entry-point information and exit-point information of each application and calling-relationship information will be extremely huge volumes of information, so that the overhead of the system will be generated considerably. Therefore, operation in which traces are always collected in case of unanticipated situation is practically impossible, and the only possible operation is, for example, collecting traces after a trouble has occurred to the extent that effects on the system will be as small as possible, so that it takes a long time to seek out the cause of a bottleneck. Also, in conventional trace processing method, operation in the distributed application system is complicated. In FIG. 2, is traces are collected by targeting the distributed application system, switches 208, 210 and 212 for indicating whether traces exist or not are equipped in the system or each application on each server 200, 202 and 204. Therefore, if traces are collected after a trouble has occurred, a manager has to turn on many switches and has to decide which switch should be turned on, so that the manager is forced into very complicated operation.